System and method for detecting portability of sentiment analysis system based on changes in a sentiment confidence score distribution

ABSTRACT

Embodiments of the present invention provide a system that that can be used to determine whether a sentiment analysis model is portable between two data sets. During operation, the system analyzes the text of a respective review in a data set (e.g., a set of reviews) using the sentiment analysis model to determine a sentiment expressed in the review. The system then computes a confidence score, which indicates the accuracy of a respective sentiment. The system subsequently determines a confidence score distribution for various sentiments, as determined by the sentiment analysis model. The system determines the significance of changes between the confidence score distribution and a benchmark confidence score distribution, which is associated with a benchmark data set for which the sentiment analysis model yields a high accuracy. The system can then determine whether the sentiment analysis model is portable to the data set based on the significance of changes.

RELATED APPLICATIONS

This application is a continuation application and claims the benefitand priority to the U.S. patent application Ser. No. 16/819,723 filed onMar. 16, 2020, which claims the benefit and priority to the U.S. patentapplication Ser. No. 15/588,503 filed on May 5, 2017 (now U.S. Pat. No.10,592,606), all of which are incorporated herein by their entirety.

BACKGROUND Field

This disclosure is generally related to sentiment analysis. Morespecifically, this disclosure is related to a method and system fordetermining portability of a sentiment analysis system.

Related Art

With the advancement of the computer and network technologies, variousoperations performed by users from different applications lead toextensive use of web services. This proliferation of the Internet andInternet-based user activity continues to create a vast amount ofdigital content. For example, multiple users may provide reviews (e.g.,fill out surveys) about a business entity (e.g., a hotel or arestaurant) via different applications, such as mobile applicationsrunning on different platforms, as well as web-interfaces running ondifferent browsers in different operating systems. Furthermore, usersmay also use different social media outlets to post their reviews aboutthe business entity.

An application server for the business entity may store the reviews in alocal storage device. Machine learning techniques can be used on thereviews to obtain the sentiment from the reviews. Sentiment analysisinvolves determining whether the text of a review expresses positive,negative, neutral, or mixed sentiments. Such sentiment analysistypically uses a historic data set for training a sentiment analysismodel. For example, a sentiment analysis model can be trained using atraining data set that has been labeled by a user (e.g., the sentimentshave been identified by the user). The trained model learns theassociations between various language patterns and the correspondingsentiments in the training data set. The trained model is then used toanalyze subsequent new data sets. When the trained model is used toanalyze new data sets similar to the training data set, the model canachieve high accuracy.

Although a number of methods are available for sentiment analysis, someproblems still remain in determining whether a sentiment analysis modelis portable to another data set in the same domain or to a new domain.

SUMMARY

Embodiments of the present invention provide a system that can be usedto determine whether a sentiment analysis model can be portable betweentwo data sets. During operation, the system analyzes the text of arespective review in a data set (e.g., a set of reviews) using thesentiment analysis model to determine a sentiment expressed in thereview. The system then computes a confidence score, which indicates anaccuracy of a respective sentiment. The system subsequently determines aconfidence score distribution for various sentiments, as determined bythe sentiment analysis model. The system further determines thesignificance of changes between the confidence score distribution and abenchmark confidence score distribution, which is associated with abenchmark data set for which the sentiment analysis model yields a highaccuracy. The system can then determine whether the sentiment analysismodel is portable to the data set based on the significance of changes.

If the significance of changes is greater than or equal than aportability threshold, the system can determine that the sentimentanalysis model is portable to the data set, thereby indicating that thesentiment analysis model can yield a high accuracy for the data set.

The benchmark data set can include a set of reviews over a period oftime for a business entity. The data set can be one of: (i) a set ofreviews over a subsequent period of time for the same business entity;(ii) a set of reviews for a another business entity in a same industryas the business entity; and (iii) a set of reviews for a anotherbusiness entity in a different industry than the business entity.

The portability threshold can be a percentage of variation in accuracythat the business entity can tolerate.

On the other hand, if the significance of changes is less than theportability threshold, the system can determine that the sentimentanalysis model needs retraining for the data set, and can indicate thatthe sentiment analysis model is not portable to the data set. In someembodiments, the system can determine whether the significance ofchanges is less than the portability threshold by determining a recallof a set of p-values obtained from applying a Kolmogorov-Smirnov (K-S)test to the confidence score distribution and the benchmark confidencescore distribution, and comparing the recall with the portabilitythreshold.

It should be noted that the system can use the benchmark data set totrain the sentiment analysis model.

The system can also determine respective median scores for theconfidence score distribution and the benchmark confidence scoredistribution. If the median score for the confidence score distributionis higher than the median score for the benchmark confidence scoredistribution, the system can set the data set as a benchmark data setfor subsequent application of the sentiment analysis model.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1A illustrates an exemplary portability analysis system, inaccordance with an embodiment of the present invention.

FIG. 1B illustrates exemplary components of a portability analysissystem, in accordance with an embodiment of the present invention.

FIG. 2 presents a flowchart illustrating a method of a portabilityanalysis system determining portability of a sentiment analysis model,in accordance with an embodiment of the present invention.

FIG. 3 illustrates an exemplary portability analysis of a sentimentanalysis model, in accordance with an embodiment of the presentinvention.

FIG. 4A presents a flowchart illustrating a method for determining aconfidence score distribution of a benchmark data set using a sentimentanalysis model, in accordance with an embodiment of the presentinvention.

FIG. 4B presents a flowchart illustrating a method for determining aconfidence score distribution of a new data set using a sentimentanalysis model, in accordance with an embodiment of the presentinvention.

FIG. 4C presents a flowchart illustrating a method for determiningwhether a sentiment analysis model is portable from a benchmark data setto a new data set, in accordance with an embodiment of the presentinvention.

FIG. 5 presents a flowchart illustrating a method for determining a newbenchmark data set for a sentiment analysis model, in accordance with anembodiment of the present invention.

FIG. 6 illustrates an exemplary computer and communication system thatfacilitates a portability analysis system, in accordance with anembodiment of the present invention.

In the figures, like reference numerals refer to the same figureelements.

DETAILED DESCRIPTION

The following description is presented to enable any person skilled inthe art to make and use the embodiments, and is provided in the contextof a particular application and its requirements. Various modificationsto the disclosed embodiments will be readily apparent to those skilledin the art, and the general principles defined herein may be applied toother embodiments and applications without departing from the spirit andscope of the present disclosure. Thus, the present invention is notlimited to the embodiments shown, but is to be accorded the widest scopeconsistent with the principles and features disclosed herein.

Overview

In embodiments of the present invention, the problem of automaticallydetermining portability of a sentiment analysis model is solved byproviding a system that automatically and proactively performsportability analysis for the sentiment analysis model. Portability of asentiment analysis model indicates whether the sentiment analysis modelthat is trained for one set of user reviews can be applied to anotherset of user reviews.

Due to ease of access via the Internet, a large number of users canprovide reviews about a business entity using one or more distributedservices (e.g., TripAdvisor, Facebook, Twitter, Yelp, etc.). Such areview can include a textual description of the user's sentiments. Asentiment analysis model analyzes the text of the review to determinethe sentiment expressed in the review (e.g., positive, negative,neutral, or mixed sentiment). Typically, a sentiment analysis model istrained using a training data set, for which the sentiments have alreadybeen identified. The trained model is then used to determine usersentiments in subsequent new data sets.

However, a new data set typically changes and evolves from the trainingdata set (e.g., a new phenomenon may appear). If the new data setdeviates significantly from the training data set, the accuracy of thesentiment analysis model may decrease significantly. With existingtechnologies, such a decrement in accuracy can be detected based on areactive intervention. If the administrator observes that the sentimentanalysis model has not accurately determined the sentiments, theadministrator can manually parse the review to determine the sentimentexpressed in the review and compare the result with the sentimentsprovided by the sentiment analysis model.

As a result, the sentiment analysis model requires the administrator toreact to the decrement in accuracy and determine whether the model isportable to the subsequent data sets. In the same way, if a sentimentanalysis model is trained for one business entity in one domain, theadministrator may wish to deploy the model for another business entityin the same domain or a business entity in another domain. For example,a sentiment analysis model trained using hotel reviews of a hotel chaincan be portable to the reviews of another hotel chain or the restaurantreviews of a restaurant chain. Determining portability for the model,thus, relies on the administrator's manual intervention. This processcan be tedious and time-consuming, and may not be scalable.

To solve this problem, embodiments of the present invention provide aportability analysis system that can proactively and automaticallydetect portability of a sentiment analysis model, thereby ensuring theperformance and quality of the determinations based on the sentimentanalysis model for the new data sets. During operation, the sentimentanalysis model is trained with a benchmark data set. This benchmark dataset can be an initial training data set (e.g., a data set representingreviews in a first quarter time-period for a business entity).

In addition to determining the sentiment from the text of a review, themodel can also calculate a confidence score for the determinedsentiment. This confidence score indicates how accurate the determinedsentiment is. Typically, a high confidence score indicates highaccuracy. The system generates a confidence score distribution based onthe confidence scores obtained from the benchmark data set. Thisconfidence score distribution can be referred to as the benchmarkconfidence score distribution.

For a new data set, the sentiment analysis model determines thesentiment as well as the corresponding confidence scores. The new dataset can be associated with a subsequent time period for the samebusiness entity (e.g., a data set representing reviews in a secondquarter), a different business entity in the same domain, or a businessentity in a different domain (e.g., a data set representing reviews fora different industry). The system generates a new confidence scoredistribution for the new data set. The system then compares thebenchmark and new confidence score distributions to determine thedifferences (or changes) between the distributions.

In some embodiments, the system can use a Kolmogorov-Smirnov test (K-Stest) to detect whether the determined differences are significant. Ifthe differences are not significant (e.g., less than a portabilitythreshold), the system can determine that the sentiment analysis modelis portable to the subsequent time period or the new businessentity/domain. In this way, the portability analysis system facilitatesautomatic detection of portability for the sentiment analysis modelwithout intervention from an administrator, thereby providing anautomatic approach comparable to human's performance.

Portability Analysis System

FIG. 1A illustrates an exemplary portability analysis system, inaccordance with an embodiment of the present invention. In this example,a large number of users 122, 124, and 126 of a business entity providereviews 152, 154, and 156, respectively, about the business entity via avariety of computing devices 132, 134, and 136, respectively. Here,users 122, 124, and 126 can be considered as reviewers for the businessentity. Examples of a review include, but are not limited to, a surveywith numerical indicators, a social media post, and a review posted on awebsite. Such a business entity can be an entity in the hospitalitybusiness (e.g., a hotel, an event management company, a theme park, atransportation service provider, a cruise line, etc.).

These computing devices are coupled via a network 140, which can be alocal or wide area network, to an application server 142 that provides adistributed service (e.g., TripAdvisor, Facebook, Twitter, Yelp, etc.).It should be noted that these reviews can be hosted on different serversassociated with the corresponding service. The business entity canmaintain a business server 144 coupled to network 140. Business server144 can store the review information of the business entity provided bythe distributed service. Such review information can include reviews ofthe business entity over a period of time (e.g., on a quarterly basis).

The business entity can run a sentiment analysis model 102 on ananalysis server 146. During operation, model 102 is trained using atraining data set with identified sentiments. When model 102 is trained,model 102 is used to determine user sentiments in a new data set.Suppose that a new data set 150 includes reviews 152, 154, and 156. Dataset 150 can be a data set for a subsequent time period of the trainingdata set or for a different business entity/domain than the trainingdata set. Analysis server 146 can obtain data set 150 from businessserver 144 and store in a local storage device 148. Model 102 analyzesthe text of reviews 152, 154, and 156 to determine the sentimentexpressed in these reviews. For example, model 102 can determine howdifferent phrases in the reviews correspond to one or more sentimentsexpressed in the review. Model 102 can generate a respective tag forreviews 152, 154, and 156. The tag can indicate whether thecorresponding review expresses positive, negative, neutral, or mixedsentiment.

However, data set 150 can change and evolve from the training data set.If data set 150 deviates significantly from the training data set, theaccuracy of model 102 may decrease significantly. With existingtechnologies, such a decrement in accuracy can be detected based on anadministrator 128's reactive intervention. Administrator 128 can be anadministrator of the business entity. If administrator 128 observes thatthe sentiments provided by model 102 have not accurately reflected thesentiments in data set 150, administrator 128 manually parses reviews152, 154, and 156 to determine the sentiments expressed in data set 150and compare the result with the sentiments provided by model 102.

As a result, model 102 requires administrator 128 to react to thedecrement in accuracy and determine whether model 102 is portable todata set 150. For example, if reviews 152, 154, and 156 are restaurantreviews, and model 102 is trained using hotel reviews, administrator 128needs to manually determine whether model 102 can be used to determinethe sentiments in reviews 152, 154, and 156. Determining portability formodel 102, thus, relies on administrator 128's manual intervention. Thisprocess can be tedious and time-consuming, and may not be scalable.

To solve this problem, embodiments of the present invention provide aportability analysis system 160 that proactively and automaticallydetects portability of sentiment analysis model 102. System 160 ensuresthe performance and quality of the determinations based on model 102 fordata set 150. System 160 can run on analysis server 146. It should benoted that analysis server 146 and business server 144 can be co-locatedin the same physical device or be coupled to each other via network 140.In some embodiments, sentiment analysis model 102 is part of portabilityanalysis system 160. For example, portability analysis system 160 caninclude a determination module 162, which includes sentiment analysismodel 102.

During operation, model 102 is trained with a benchmark data set. Thisbenchmark data set can be the initial training data set. In addition todetermining the sentiments, model 102 can also calculate respectiveconfidence scores for the determined sentiments. System 160 obtains theconfidence scores (e.g., from one or more network packets) and storesthem in a local storage device 148. System 160 generates a confidencescore distribution based on the confidence scores and stores thedistribution in storage device 148. This confidence score distributioncan be referred to as the benchmark confidence score distribution. Model102 also determines the corresponding confidence scores for thesentiments determined for data set 150. System 160 generates a newconfidence score distribution for data set 150. A comparison module 164of system 160 then compares the benchmark and new confidence scoredistributions to determine the differences (or changes) between thedistributions.

In some embodiments, system 160 uses a K-S test to detect whether thedetermined differences are significant. If the differences are notsignificant (e.g., less than a portability threshold), system 160determines that model 102 is portable to data set 150. This portabilitythreshold can be predetermined by administrator 128 or derived fromempirical data. On the other hand, if system 160 determines that model102 is not portable to data set 150, a recommendation module 168 ofsystem 160 proactively recommends a retraining for model 102 for dataset 150 to administrator 128. In this way, system 160 facilitatesautomatic detection of portability for model 102 without interventionfrom administrator 128, thereby providing an automatic approachcomparable to a human's performance.

In some embodiments, system 160 further includes an update module 166,which updates the benchmark data set. For example, if the medianconfidence score of data set 150 is more than the median confidencescore of the benchmark data set, update module 166 sets data set 150 asthe benchmark data set. Adjusting the benchmark data set based on themedian confidence score ensures that system 160 maintains thedistribution with the highest accuracy as the benchmark data set.

FIG. 1B illustrates exemplary components of a portability analysissystem, in accordance with an embodiment of the present invention. Inthis example, sentiment analysis model 102 is incorporated withdetermination module 162. Model 102 can include a sentiment predictionmechanism 172, which analyzes the text of review 152 in data set 150 todetermine (or predict) the sentiment expressed in review 152. Model 102further includes a confidence score generation mechanism 174, whichcalculates a confidence score for the determined sentiment for review152. In the same way, sentiment prediction mechanism 172 determines thesentiment expressed in reviews 154 and 156, and confidence scoregeneration mechanism 174 calculates a respective confidence score forthe determined sentiments for reviews 154 and 156.

Confidence score generation mechanism 174 then calculates a confidencescore distribution 182 based on the calculated confidence scores andprovides confidence score distribution 182 to comparison module 164.Comparison module 164 also obtains a benchmark confidence scoredistribution 184 (e.g., from storage device 148, as described inconjunction with FIG. 1A). Comparison module 164 includes a median scoreverification mechanism 176, which determines whether the medianconfidence score of data set 150 is more than the median confidencescore of the benchmark data set. If so, comparison module 164 providesan update indicator to update module 166, which, in turn, sets data set150 as a new benchmark data set 190.

Comparison module 164 further includes a statistics verificationmechanism 178, which compares confidence score distributions 182 and 184to determine the differences (or changes) between distributions 182 and184. Statistics verification mechanism 178 uses a K-S test to determinethe significance of the differences. Based on the significance of thedifferences, recommendation module 168 provides a recommendation 180 toadministrator 128 indicating whether model 102 is portable to data set150. For example, if the differences are less than a portabilitythreshold (e.g., a 5% threshold), recommendation 180 indicates thatmodel 102 is portable to data set 150. This threshold can bepredetermined by administrator 128 or derived from empirical data. Onthe other hand, if system 160 determines that model 102 is not portableto data set 150, recommendation module 168 proactively indicates inrecommendation 180 that a retraining for model 102 for data set 150 isneeded.

Portability Analysis

FIG. 2 presents a flowchart illustrating a method of a portabilityanalysis system determining portability of a sentiment analysis model,in accordance with an embodiment of the present invention. Duringoperation, the system uses data set 150 and benchmark confidence scoredistribution 184 to determine the portability of sentiment analysismodel 102. The system executes sentiment analysis model on data set 150to obtain confidence score distribution 182 (operation 202). The systemdetermines the significance of differences between confidence scoredistribution 182 and benchmark confidence score distribution 184(operation 204). The system then generates recommendation 180 indicatingportability of sentiment analysis model 102 based on the significance ofdifferences (operation 206). The system can display a recommendation toadministrator 128 indicating the portability of sentiment analysis model102.

FIG. 3 illustrates an exemplary portability analysis of a sentimentanalysis model, in accordance with an embodiment of the presentinvention. Portability analysis system 160 maintains portabilityanalysis 300, which represents the significance of changes (e.g., K-Stest results) for a business entity's data set over time (or acrossbusiness entities/domains). System 160 compares the changes between newand benchmark confidence score distributions 182 and 184, respectively,with a statistical significance indicator. In some embodiments, thestatistical significance indicator is the p-value from the K-S test.System 160 determines an accuracy threshold such that differences belowthe accuracy threshold yield a recommendation indicating that sentimentanalysis model 102 is not portable. System 160 can determine thisaccuracy threshold based on empirical data.

System 160 also determines a threshold for a statistical significance,which can be determined based on the p-value from the K-S test. System160 then determines the true-positive data points, such as data point302, from the comparison (left-bottom corner of portability analysis300). For a true-positive data point, the p-value indicates statisticalsignificance with accuracy below the accuracy threshold. System 160further determines the false-positive data points, such as data point304, from the comparison (right-bottom corner of portability analysis300). For a false-positive data point, the p-value indicatesnon-significance with accuracy below the accuracy threshold. Based onthe true-positive and false-positive data points, system 160 determinesthe recall, which is the percentage of model 102 able to detect datapoints with statistical significance among all data points below theaccuracy threshold. If the recall is below a portability threshold,system 160 recommends portability for model 102.

Operations

FIG. 4A presents a flowchart illustrating a method 400 for determining aconfidence score distribution of a benchmark data set using a sentimentanalysis model, in accordance with an embodiment of the presentinvention. During operation, a portability analysis system obtains abenchmark data set (operation 402) and applies the sentiment analysismodel on the benchmark data set to determine the sentiment for arespective review in the benchmark data set (operation 404). The systemfurther determines the confidence score for a respective sentimentdetermined for the benchmark data set (operation 406). The system thendetermines and stores the benchmark confidence score distribution forthe benchmark data set (operation 408).

FIG. 4B presents a flowchart illustrating a method 430 for determining aconfidence score distribution of a new data set using a sentimentanalysis model, in accordance with an embodiment of the presentinvention. During operation, a portability analysis system obtains a newdata set associated with a different time period of the same businessentity, another business entity in the same domain (e.g., anotherbusiness entity in the same industry), or a different domain (e.g.,another business entity in another industry) (operation 432). The systemapplies the sentiment analysis model on the new data set to determinethe sentiment for a respective review in the new data set (operation434). The system further determines the confidence score for arespective sentiment determined for the new data set (operation 436).The system then determines the new confidence score distribution for thebenchmark data set (operation 438).

FIG. 4C presents a flowchart illustrating a method 450 for determiningwhether a sentiment analysis model is portable from a benchmark data setto a new data set, in accordance with an embodiment of the presentinvention. During operation, a portability analysis system compares thenew and benchmark confidence score distributions (operation 452). Thesystem then obtains the accuracy threshold and the significant threshold(operation 454) and determines the recall for the comparison (operation456). The system checks whether the recall is less than the portabilitythreshold (operation 458). If the recall is less than the portabilitythreshold, the system determines that the sentiment analysis model isportable to the new data set (operation 460). Otherwise, the systemdetermines that the sentiment analysis model needs retraining for thenew data set (operation 462).

FIG. 5 presents a flowchart illustrating a method 500 for determining anew benchmark data set for a sentiment analysis model, in accordancewith an embodiment of the present invention. During operation, aportability analysis system determines the respective median confidencescores for the benchmark and the new data sets (operation 502). Thesystem compares the determined median values (operation 504). The systemchecks whether the new median confidence score is higher than the medianconfidence score of the benchmark data set (operation 506). If the newmedian confidence score is higher, the system allocates the new data setas the benchmark data set (operation 508).

Exemplary Computer and Communication System

FIG. 6 illustrates an exemplary computer and communication system thatfacilitates a portability analysis system, in accordance with anembodiment of the present invention. A computer and communication system602 includes a processor 604, a memory 606, and a storage device 608.Memory 606 can include a volatile memory (e.g., RAM) that serves as amanaged memory, and can be used to store one or more memory pools.Furthermore, computer and communication system 602 can be coupled to adisplay device 610, a keyboard 612, and a pointing device 614. Storagedevice 608 can store an operating system 616, a portability analysissystem 618, and data 632.

Portability analysis system 618 can include instructions, which whenexecuted by computer and communication system 602, can cause computerand communication system 602 to perform the methods and/or processesdescribed in this disclosure. Portability analysis system 618 includesinstructions for determining the sentiment expressed in the text of arespective user review in a data set and a corresponding confidencescore for the sentiment (determination module 620). Portability analysissystem 618 can also include instructions for determining a confidencescore distribution for the data set (determination module 620).

Portability analysis system 618 further includes instructions forcomparing a new confidence score distribution with a benchmarkconfidence score distribution (e.g., for calculating recall from a K-Stest) (comparison module 622). Portability analysis system 618 can alsoinclude instructions for updating a benchmark data set with a new dataset based on respective median scores (update module 624). Portabilityanalysis system 618 can include instructions for facilitating acommendation indicating whether a sentiment analysis model is portableto a new data set based on whether the recall is less than a portabilitythreshold (recommendation module 626). Portability analysis system 618can include instructions for displaying, via display device 610 using agraphical or textual interface, a recommendation to an administratorindicating the portability of the sentiment analysis model(recommendation module 626).

Portability analysis system 618 can also include instructions forexchanging information with other devices (communication module 628).Data 632 can include any data that is required as input or that isgenerated as output by the methods and/or processes described in thisdisclosure. Data 632 can include one or more of: a benchmark confidencescore distribution, a new confidence score distribution, a benchmarkdata set, and a new data set.

The data structures and code described in this detailed description aretypically stored on a computer-readable storage medium, which may be anydevice or medium that can store code and/or data for use by a computersystem. The computer-readable storage medium includes, but is notlimited to, volatile memory, non-volatile memory, magnetic and opticalstorage devices such as disk drives, magnetic tape, CDs (compact discs),DVDs (digital versatile discs or digital video discs), or other mediacapable of storing computer-readable media now known or later developed.

The methods and processes described in the detailed description sectioncan be embodied as code and/or data, which can be stored in acomputer-readable storage medium as described above. When a computersystem reads and executes the code and/or data stored on thecomputer-readable storage medium, the computer system performs themethods and processes embodied as data structures and code and storedwithin the computer-readable storage medium.

Furthermore, the methods and processes described above can be includedin hardware modules or apparatus. The hardware modules or apparatus caninclude, but are not limited to, application-specific integrated circuit(ASIC) chips, field-programmable gate arrays (FPGAs), dedicated orshared processors that execute a particular software module or a pieceof code at a particular time, and other programmable-logic devices nowknown or later developed. When the hardware modules or apparatus areactivated, they perform the methods and processes included within them.

The foregoing descriptions of embodiments of the present invention havebeen presented for purposes of illustration and description only. Theyare not intended to be exhaustive or to limit the present invention tothe forms disclosed. Accordingly, many modifications and variations willbe apparent to practitioners skilled in the art. Additionally, the abovedisclosure is not intended to limit the present invention. The scope ofthe present invention is defined by the appended claims.

What is claimed is:
 1. A computer-implemented method for facilitatingportability analysis for a sentiment analysis model, the methodcomprising: determining a first distribution of confidence scores for afirst data set by applying a sentiment analysis model to a plurality ofreviews in the first data set; comparing the first distribution ofconfidence scores to a benchmark distribution of confidence scores,wherein a benchmark data set associated with the benchmark distributionis different from the first data set; in response to the comparing,determining a recall based on an accuracy threshold and a significantthreshold; and comparing the recall to a portability threshold and inresponse thereto determining whether the sentiment analysis modelassociated with the benchmark data set is portable to the first dataset.
 2. The computer-implemented method of claim 1 further comprisingdisplaying, via a display device, a recommendation indicating whetherthe sentiment analysis model is portable to the first data set.
 3. Thecomputer-implemented method of claim 1, wherein the sentiment analysismodel is portable to the first data set if the recall is less than theportability threshold.
 4. The computer-implemented method of claim 1further comprising retraining the sentiment analysis model if thesentiment analysis model is determined to be not portable to the firstdata set.
 5. The computer-implemented method of claim 1, wherein theaccuracy threshold is based on empirical data.
 6. Thecomputer-implemented method of claim 1, wherein the significancethreshold is based on a p-value obtained from a K-S test.
 7. Thecomputer-implemented method of claim 1, wherein the recall is detectingdata points with statistical significance among all data points belowthe accuracy threshold.
 8. The computer-implemented method of claim 1,wherein the recall is further based on a true-positive data point and afalse-negative data point.
 9. The computer-implemented method of claim1, wherein a review from the plurality of reviews comprises text. 10.The computer-implemented method of claim 1, further comprising:generating a tag for a review from the plurality of reviews to indicatewhether the review expresses positive, negative, neutral, or mixedsentiment.
 11. The computer-implemented method of claim 1, wherein thebenchmark data set comprises a set of reviews over a first period oftime for a business entity, and wherein the first data set is a set ofreviews over a second period of time for the same business entity. 12.The computer-implemented method of claim 1, wherein the benchmark dataset comprises a set of reviews over a first period of time for abusiness entity, and wherein the first data set is a set of reviews fora second business entity in a same industry as the business entity. 13.The computer-implemented method of claim 1, wherein the benchmark dataset comprises a set of reviews over a first period of time for abusiness entity, and wherein the first data set is a set of reviews fora third business entity in a different industry than the businessentity.
 14. A computer system for non-parametric correlation analysis,the system comprising: a processor; and a storage device storinginstructions that when executed by the processor causes the processor toperform a method, the method comprising: determining a firstdistribution of confidence scores for a first data set by applying asentiment analysis model to a plurality of reviews in the first dataset; comparing the first distribution of confidence scores to abenchmark distribution of confidence scores, wherein a benchmark dataset associated with the benchmark distribution is different from thefirst data set; in response to the comparing, determining a recall basedon an accuracy threshold and a significant threshold; and comparing therecall to a portability threshold and in response thereto determiningwhether the sentiment analysis model associated with the benchmark dataset is portable to the first data set.
 15. The computer-implementedmethod of claim 14 further comprising displaying, via a display device,a recommendation indicating whether the sentiment analysis model isportable to the first data set.
 16. The computer-implemented method ofclaim 14, wherein the sentiment analysis model is portable to the firstdata set if the recall is less than the portability threshold.
 17. Thecomputer-implemented method of claim 14 further comprising retrainingthe sentiment analysis model if the sentiment analysis model isdetermined to be not portable to the first data set.
 18. Thecomputer-implemented method of claim 14, wherein the accuracy thresholdis based on empirical data.
 19. The computer-implemented method of claim14, wherein the significance threshold is based on a p-value obtainedfrom a K-S test.
 20. The computer-implemented method of claim 14,wherein the recall is detecting data points with statisticalsignificance among all data points below the accuracy threshold.
 21. Thecomputer-implemented method of claim 14, wherein the recall is furtherbased on a true-positive data point and a false-negative data point. 22.The computer-implemented method of claim 14, wherein a review from theplurality of reviews comprises text.
 23. The computer-implemented methodof claim 14, further comprising: generating a tag for a review from theplurality of reviews to indicate whether the review expresses positive,negative, neutral, or mixed sentiment.
 24. The computer-implementedmethod of claim 14, wherein the benchmark data set comprises a set ofreviews over a first period of time for a business entity, and whereinthe first data set is a set of reviews over a second period of time forthe same business entity.
 25. The computer-implemented method of claim14, wherein the benchmark data set comprises a set of reviews over afirst period of time for a business entity, and wherein the first dataset is a set of reviews for a second business entity in a same industryas the business entity.
 26. The computer-implemented method of claim 14,wherein the benchmark data set comprises a set of reviews over a firstperiod of time for a business entity, and wherein the first data set isa set of reviews for a third business entity in a different industrythan the business entity.